Papers by Harish Tayyar Madabushi
Pre-Trained Language Models Represent Some Geographic Populations Better than Others (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have focused on measuring the degree to which pre-trained language models capture purely linguistic knowledge and reasoning abilities and world knowledge. |
| Approach: | They use geography to demarcate different populations around the world and comparable corpora to measure how well two families of LLMs perform across these different populations. |
| Outcome: | The results show that pre-trained models perform better for some populations than others. |
CxGBERT: BERT meets Construction Grammar (2020.coling-main)
Copied to clipboard
| Challenge: | lexico-semantic elements capture a large amount of linguistic information, but they do not capture all information contained in text. |
| Approach: | They propose to use BERT to train a model that uses a deep bidirectional transformer to capture a significant amount of lexico-semantic information. |
| Outcome: | The proposed model captures lexico-semantic information, but it is redundantly encoded in lexical information. |
Are Emergent Abilities in Large Language Models just In-Context Learning? (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models have been claimed to acquire certain capabilities without having been specifically trained on them. |
| Approach: | They propose a theory that explains emergent abilities by taking into account their potential confounding factors and rigorously substantiate this theory through over 1000 experiments. |
| Outcome: | The proposed theory proves that emergent abilities are not truly emergental, but result from a combination of in-context learning, model memory, and linguistic knowledge. |
Multi-class Hierarchical Question Classification for Multiple Choice Science Exams (2020.lrec-1)
Copied to clipboard
Dongfang Xu, Peter Jansen, Jaycie Martin, Zhengnan Xie, Vikas Yadav, Harish Tayyar Madabushi, Oyvind Tafjord, Peter Clark
| Challenge: | Prior work has demonstrated that question classification (QC) can help answer a question more accurately. |
| Approach: | They propose to use a large dataset for question classification (QC) that contains 7,787 science exam questions paired with detailed classification labels from a fine-grained hierarchical taxonomy of 406 problem domains to train a BERT-based model. |
| Outcome: | The proposed model achieves a large (+0.12 MAP) gain while also achieving state-of-the-art performance on benchmark open-domain and biomedical QC datasets. |
A Construction Grammar Corpus of Varying Schematicity: A Dataset for the Evaluation of Abstractions in Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been developed without a theoretical framework . evaluating and improving LLMs will benefit from theoretical frameworks that enable comparison of structures of human language and model of language built up by LLM. |
| Approach: | They propose to use a construction grammar schema corpus to compare human grammar to LLMs' model of language. |
| Outcome: | The proposed corpus shows that even the largest LLMs are limited to more substantive constructions and do not recognize similarity of purely schematic constructions. |
AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets are limited to providing the degree of idiomaticity of expressions along with the literal and, where applicable, (a single) non-literal interpretation of MWEs. |
| Approach: | They propose to use a dataset to test the effectiveness of a language model in generating representations of sentences containing idioms. |
| Outcome: | The proposed model performs reasonably well on the one-shot and few-shot scenarios, but there is scope for improvement in the zero-shot scenario. |
Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to generate multiple independent CoTs, combining them through ensembling or other post-hoc strategies, have been shown to be effective in boosting performance. |
| Approach: | They propose a method where LLMs are fine-tuned to generate a sequence of Diverse Chains of Thought (DCoT) within a single inference step. |
| Outcome: | The proposed model can generate multiple chains of thought within a single inference step without external feedback. |
Code-Mixed Probes Show How Pre-Trained Models Generalise on Code-Switched Text (2024.lrec-main)
Copied to clipboard
| Challenge: | Code-switching is a prevalent linguistic phenomenon in which multilingual individuals seamlessly alternate between languages. |
| Approach: | They propose to use pre-trained language models to generalise to code-switched text . they use a dataset of well-formed naturalistic code-witched texts and parallel translations into the source languages to examine their results. |
| Outcome: | The proposed model generalises to code-switched text, shedding light on their ability to generalise representations to CS corpora. |
Improving Tokenisation by Alternative Treatment of Spaces (2022.emnlp-main)
Copied to clipboard
| Challenge: | Subword tokenisation is a key initial step in processing natural language . it uses a number of different methods to tokenise text, including a stringsearching technique and a word-matching technique. |
| Approach: | They propose to use a vocabulary-based approach to tokenise text using a numerical ID and a mathematical function to manipulate it. |
| Outcome: | The method is based on a set of training data and learning from it to build a vocabulary and tokenise it at inference time using this vocabulary and learnt parameters. |
Cost-Sensitive BERT for Generalisable Sentence Classification on Imbalanced Data (D19-50)
Copied to clipboard
| Challenge: | Popular NLP tasks such as sentiment analysis and event extraction from social media are examples of imbalanced classification problems. |
| Approach: | They propose a method to generalise on dissimilar training and test data using a measure of similarity between datasets. |
| Outcome: | The proposed method achieves the second highest score on sentence-level propaganda classification. |
SpeciaLex: A Benchmark for In-Context Specialized Lexicon Learning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Specialized lexicons are collections of words with associated constraints such as special definitions, specific roles, and intended target audiences. |
| Approach: | They propose a benchmark to evaluate a language model’s ability to follow specialized lexicon-based constraints across 18 diverse subtasks with 1,785 test instances covering core tasks of Checking, Identification, Rewriting, and Open Generation. |
| Outcome: | The proposed model can follow specialized lexicon-based constraints across 18 diverse subtasks with 1,785 test instances covering core tasks Checking, Identification, Rewriting, and Open Generation. |
Standardize: Aligning Language Models with Expert-Defined Standards for Content Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Domain experts in engineering, healthcare, and education follow strict standards for producing quality content. |
| Approach: | They propose a retrieval-style in-context learning-based framework to guide large language models to align with expert-defined standards. |
| Outcome: | The proposed framework shows that models can gain 45% to 100% increase in precise accuracy across open and commercial LLMs evaluated. |
UniversalCEFR: Enabling Open Multilingual Research on Language Proficiency Assessment (2025.emnlp-main)
Copied to clipboard
Joseph Marvin Imperial, Abdullah Barayan, Regina Stodden, Rodrigo Wilkens, Ricardo Muñoz Sánchez, Lingyun Gao, Melissa Torgbi, Dawn Knight, Gail Forey, Reka R. Jablonkai, Ekaterina Kochmar, Robert Joshua Reynolds, Eugénio Ribeiro, Horacio Saggion, Elena Volodina, Sowmya Vajjala, Thomas François, Fernando Alva-Manchego, Harish Tayyar Madabushi
| Challenge: | Language proficiency research plays a central role in education and often intersects with advances in linguistics and AI. |
| Approach: | They propose a multilingual multidimensional dataset of texts annotated according to the CEFR scale in 13 languages. |
| Outcome: | The proposed dataset supports linguistic features and pretrained models in multilingual CEFR level assessment. |
Integrating Question Classification and Deep Learning for improved Answer Selection (C18-1)
Copied to clipboard
| Challenge: | Question Answering (QA) is the task of automatically generating answers to questions posed in natural language. |
| Approach: | They propose a system for Answer Selection that integrates fine-grained Question Classification with a Deep Learning model designed for Answer selection. |
| Outcome: | The proposed system outperforms the current state of the art in all variations except one . the proposed system improves QA by reducing the search space of potential answers . |
Abstraction not Memory: BERT and the English Article System (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained models are the state of the art in linguistics. |
| Approach: | They compare the performance of pre-trained and native English language models on the task of article prediction set up as a three way choice (a/an, the, zero) they argue that BERT captures a high level generalisation of article use akin to human intuition. |
| Outcome: | The proposed model outperforms humans on the linguistically interesting task of article prediction. |